In the case of digitizing a signal of a moving picture to record and transmit digital data thus obtained, because its data quantity becomes very large, coding (compression) is implemented to the data. As a representative coding system, there is a motion compensating predictive coding.
The principle of motion compensating prediction is shown in FIG. 9. In FIG. 9, the motion compensating predictive coding is a method utilizing correlation in a time base direction of a picture signal. This method predicts a currently inputted picture (picture of current frame) from a known picture signal already decoded and reproduced (picture signal of past frame) to move the known picture signal decoded and reproduced (picture signal of past frame) in correspondence with motion of signal to transmit only motion information (motion vector MV) at that time and predictive error information at that time, thus to compress information quantity required for coding.
As a representative one of motion compensating predictive coding and decoding methods as described above, there is so called MPEG (Moving Picture Expert Group) 1. This MPEG1 is a popular name of the moving picture coding system which has progressed in WG (Working Group) 11 of SC (Sub Committee) 29 of JTC (Joint Technical Committee) 1 of so called ISO (International Standardization Organization) and IEC (International Electrotechnical Commission).
MPEG1 has intraframe and interframe coding processing, and the motion compensating predictive coding is used in the case of the interframe coding processing. An example of motion compensating prediction at this time is shown in FIG. 10. In FIG. 10, in the motion compensating prediction at this time, one frame (picture) is divided into small blocks (called macro blocks (MB) and each is comprised of 16 lines.times.16 pixels) to carry out motion compensating predictive coding in units of those small blocks. In the example of FIG. 10, as motion compensating prediction in the case of the interframe coding processing, forward prediction between, e.g., past frame present in the past in point of time and current frame and backward prediction between, e.g., future frame present in future in point of time and current frame are illustrated.
A more practical example of motion compensation at this time is shown in FIG. 11. Initially, as processing of the first stage, P picture (to be coded) indicated by P in the figure jumping over several pictures is predictive-coded by using I picture indicated by I in the figure or any other P picture. Such predictive coding operation is cyclically executed. Then, as processing of the second stage, B pictures indicated by B in the figure put between P pictures, I pictures or between picture and I picture are sequentially predictive-coded from P picture or I picture before and after those B pictures. It is to be noted that I picture is intraframe coded picture which is prepared without use of motion compensation. Further, arrows (except for arrow indicating direction of time t) in FIG. 11 indicate directions of motion predictions.
For motion compensation at the MPEG1, four kinds of modes described below exist in units of macroblock. Namely, there exist:
mode of prediction from a frame present in the past in point of time as first motion compensation mode (forward predictive mode);
mode of prediction from a frame present in future in point of time as second motion compensation mode (backward predictive mode);
mode of linear prediction from both the past and future frames as third motion compensation mode (bidirectionally predictive mode) (i.e., predictive blocks from past frame and predictive blocks from future frame are caused to undergo mean value calculation every pixel); and
mode having no motion compensation as fourth motion compensation mode (i.e., intra-coded mode).
Among these four kinds of motion compensation modes, in the P picture, an optimum one of the first and fourth motion compensation modes is selected in units of macroblocks. At this time, as past frame to which reference is made at the time of motion compensating prediction with respect to picture of current frame, one frame exists. Further, in the B picture, an optimum one of the first to fourth motion compensating modes is selected in units of macroblocks. At this time, as past frame and future frame to which reference is made at the time of motion compensating prediction with respect to picture of current frame, one frames respectively exist.
In the P picture and B picture, as motion vector in units of the macroblock, one motion vector which makes reference to the past frame exists in prediction in the first motion compensating mode. In prediction in the second motion compensating mode, one motion vector which makes reference to the future frame exists. In prediction in the third motion compensating mode, motion vectors which make reference to both past and future frames respectively exist.
In the MPEG1, motion vectors delivered in units of these macroblocks are caused to undergo determination (calculation) of difference by difference determination (calculation) element shown in FIG. 12 by making use of the fact that information of motion vector given in units of macroblocks have high spatial correlation, and difference information thus obtained is transmitted.
In more practical sense, at this difference determination element, terminal 404 is supplied with motion compensation mode signals given as four kinds of the first to fourth motion compensating modes, and terminal 401 is supplied with a motion vector signal currently inputted. The currently inputted motion vector signal is sent to common terminal of switch 406 and adding input terminal of difference circuit 405, and the motion compensating mode signal is sent to switcher 403.
Selected (switched) terminal a of the switch 406 is connected to input terminal of register 402a for past frame reference vector of register group 402, and selected terminal b is connected to input terminal of register 402b for future frame reference vector of register group 402. Further, output terminal of register 402a for past frame reference vector of register group 402 is connected to selected terminal a of switch 407, and output terminal of the register 402b for future frame reference vector is connected to selected terminal b of switch 407. These switches 406 and 407 are of a structure such that any one of the selected terminals a and b is selected in accordance with register switching signal S401 outputted from switcher 403 supplied with the motion compensating mode signal. An output of the switch 407 is sent to subtracting input terminal of the difference circuit 405.
Accordingly, the difference circuit 405 takes a difference between motion vector signal S402 which has been read out through the switch 407 from register group 402 in accordance with register switching signal S401 outputted from the switcher 403 and motion vector signal from the terminal 401. A difference by the difference circuit 405 is outputted from terminal 408 as a difference motion vector from the difference determination circuit.
The motion vector signal currently inputted is overwritten into register 402a or 402b designated by register switching signal S401 through the switch 406, and is newly stored thereinto.
As stated above, in the difference determination element, motion vectors making reference to past frame and motion vectors making reference to future frame are caused to independently undergo processing for determination of difference.
A more practical example of the operation of the above-described difference determination element is shown in FIG. 13. In this figure, X-component of (X, Y) of motion vector indicates motion in a horizontal direction and Y-component thereof indicates motion in a vertical direction. Positive values are given to motions spatially in right direction and in lower direction, and negative values are given to motions in left direction and in upper direction. In this example, (0, 0) is given as an initial value at the time of determination of difference. Further, in this figure, FMV represents past frame reference motion vector, and BMV represents future frame reference motion vector. In addition, in this figure, mode (1) represents motion compensation mode of forward prediction, mode (2) represents motion compensation mode of backward prediction, and mode (3) represents motion compensation mode of bidirectional prediction.
Namely, in FIG. 13, e.g., when motion compensation mode of small block at the left end of the figure is mode (3), past frame reference motion vector FMV is (10, 10), and future frame reference motion vector BMV is (8, 8), if motion vector in small block of mode (1) adjacent on the right side is (8, 5), difference motion vector becomes equal to (-2, -5) of (8, 5)-(10, 10), and if motion vector in small block of mode (1) adjacent on the right side is (7, 6), difference motion vector becomes equal to (-1, 1) of (7, 6)-(8, 5). Further, if motion vector in small block of mode (2) adjacent on the right side is (9, 10), difference motion vector becomes equal to (1, 2) of (9, 10)-(8, 8), and if input motion vectors in small block of the mode (3) adjacent on the right side are (5, 4) and (8, 9), difference motion vectors thereof become equal to (-2, -2) of (5, 4)-(7, 6) and (-1, -1) of (8, 9)-(9, 10).
An inverse difference determination element for reconstructing motion vector from motion vector caused to be represented by difference vector, which is a configuration complementary to the difference determination element shown in FIG. 13, is shown in FIG. 14.
In more practical sense, in the inverse difference determination element, terminal 414 is supplied with motion compensation mode signals given as four kinds of the first to fourth motion compensating modes, and terminal 408 is supplied with the difference motion vector signal currently inputted. The difference motion vector signal is sent to one adding input terminal of adding circuit 409, and the motion compensation mode signal is sent to switcher 413.
Further, selected terminal a of switch 411 is connected to output terminal of register 410a for past frame reference vector of register group 410, and selected terminal b is connected to output terminal of register 410b for future frame reference vector. This switch 411 is of a structure such that any one of the selected terminals a and b is selected in accordance with register switching signal S403 outputted from switcher 413 supplied with the motion compensation mode signal. Common terminal of the switch 411 is connected to the other adding input terminal of the adding circuit 409.
Accordingly, the adding circuit 409 takes an addition of motion vector signal S404 which has been read out through the switch 411 from register group 410 in accordance with register switching signal S403 outputted from the switcher 413 and difference motion vector signal from the terminal 408. An addition output of the adding circuit 409 is outputted from terminal 415 as reconstructed motion vector signal from the inverse difference determination element.
Further, the reconstructed motion vector signal from the adding circuit 409 is sent also to common terminal of switch 412. This switch 412 is of a structure such that any one of selected terminals a and b is selected on the basis of the register switching signal S403. Selected terminal a is connected to register 410a for past frame reference vector of the register group 410 and selected terminal b is connected to register 410b for future frame reference vector.
For this reason, information of the reconstructed motion vector is overwritten, through the switch 412, into register 410a or 410b designated by the register switching signal S403, and is newly stored thereinto.
Meanwhile, since the above-mentioned MPEG1 is moving picture coding method suitable for non-interlaced picture, it cannot be said that the MPEG1 is necessarily suitable for coding of interlaced signal such as so called NTSC system or PAL system, etc. which is a television signal. This was problem.
For this reason, in recent years, there has been proposed a system of implementing efficient motion compensating predictive coding to interlaced signal in the MPEG2 as successor to the MPEG1. In this system, as shown in FIG. 15, a method of adaptively carrying out switching between unit for handling picture in frame and that in field is adopted. In FIG. 15, fr indicates frame, fi indicates field, arrow FRP indicates prediction of frame base, and arrow FIP indicates prediction of field base.
At this time, in the MPEG2, in the case where the picture structure is frame, there is adopted a method of adaptively switching whether motion compensating prediction is carried out by macroblock of frame base or macroblock of field base.
Further, in the MPEG2, there is adopted a method in which macroblock is divided into areas of 16 pixels.times.8 lines of the upper portion and the lower portion as occasion demands to independently carry out motion compensating prediction in respective areas.
At this time, the number of motion vectors given in units of macroblocks is as follows. Namely, in the previously described prediction of the first motion compensating mode, one or more motion vectors which makes reference to past frame (or field) exist; in the prediction of the second motion compensating mode, one or more motion vectors which makes reference to future frame (or field) exist; and in the prediction of the third motion compensating mode, one or more motion vectors which make reference to both frames (or fields) of past and future respectively exist. The numbers of motion vectors in respective motion compensating modes are varied in units of macroblocks.
As stated above, as the motion compensating prediction method in the MPEG2, systems more complex than the MPEG1 are proposed. Followed by this, there result increased kinds of motion vectors at that time as compared to the MPEG1. Under such circumstances, two problems exist in handling these motion vectors.
Namely, the first problem is that method used in the MPEG1 cannot be applied in encoding motion vector. For this reason, various proposals have been made. They are laid open to public inspection, e.g., on P. 32, P. 44, P. 145, P. 148 of ISO-IEC/JTC1/SC29/WG11 MPEG93/225b Test Model 4 (laid open on February 1993).
However, it cannot be said that these proposals support all the motion compensating modes, and they are therefore insufficient up to now. Further, these proposals have the drawback that they cannot solve the second problem described below.
The second problem resides in that handling of skipped macroblock is not clearly shown.
Here, skipped macroblocks in the MPEG1 are macroblocks as featured below: Namely, they are characterized in that;
first, skipped macroblock at P picture
there is no motion compensation error signal to be transmitted (condition a.sub.-- 1e),
motion compensation mode is forward prediction (condition a.sub.-- 2e), and
the value of motion vector is zero (condition a.sub.-- 3e);
secondly, skipped macroblock at B picture
there is no motion compensation error signal to be transmitted (condition b.sub.-- 1e),
motion compensation mode is the same as macroblock transmitted immediately before (condition b.sub.-- 2e), and
the value of motion vector caused to be difference vector obtained by difference determination element shown in FIG. 12 is zero (condition b.sub.-- 3e).
Further, in the decoder, skipped macroblocks will be dealt in the MPEG1 as follows: Namely, in the MPEG1, skipped macroblocks are handled in such a manner that
first, processing of skipped macroblocks at P picture is that
motion compensation error signal which has been transmitted is considered to be zero (condition a.sub.-- 1d),
motion compensation mode is caused to be forward prediction (condition a.sub.-- 2d), and
the value of motion vector is set to zero (condition a.sub.-- 3d),
secondly, processing of skipped macroblocks at B picture is that
motion compensation error signal which has been transmitted is considered to be zero (condition b.sub.-- 1d),
motion compensation mode of a macroblock received immediately before is required to be copied (condition b.sub.-- 2d), and
motion vector of a macroblock received immediately before is required to be copied (condition b.sub.-- 3d).
Simply in other words, in the MPEG1, decoding processing of skipped macroblock at P picture is copy from past picture, and decoding processing of skipped macroblocks at B picture is copy from macroblock adjacent on the left side decoded immediately before within the same frame.
As described above, in the MPEG1, with respect to macroblock which is skipped macroblock, the encoder transmits no data. As stated above, the skipped macroblock is important coding technology in view of improvement in coding efficiency. For example, in the case where even if a picture to be processed is a sequence of moving pictures, its content is a scene where still pictures (pictures having very gentle motion) are successive, most macroblocks can be dealt as skipped macroblocks. In the extreme case, there may be employed a procedure to encode the first frame within picture without motion compensation to transmit coded frame information, thus permitting macroblocks from the second frames and frames succeeding thereto to be almost handled as skipped macroblocks. In this case, great improvement of coding efficiency can be expected.
From facts as described above, method for encoding motion vector in motion compensating prediction of the previously described MPEG2 is required to be a method in which handling of skipped macroblock is taken into consideration as well.
However, in the case of methods of encoding motion vector conventionally proposed to the MPEG2, handling of skipped macroblock is not taken into consideration. For this reason, those methods disadvantageously fail to handle skipped macroblocks.
For example, an example of the problem will now be described on the basis of configuration of difference determination element of FIG. 16. This is the method described in ISO-IEC/JTC1/SC29/WG11 MPEG93/225b Test Model 4 of the previously described material.
In FIG. 16, terminal 504 is supplied with mode information of forward, backward or bidirectional prediction as mode signal of motion compensation, information indicating frame prediction or field prediction, etc. as prediction type signal which will be described later, and information indicating frame structure, field structure or progressive as picture structure signal, and is further supplied, in addition to the above, with information indicating the first field (field 1, i.e., e.g., odd field) or the second field (field 2, i.e., e.g., even field) as field motion vector parity signal, information similarly indicating the first field or the second field as predicted field parity signal, and information indicating P picture or B picture as picture coding type signal. In addition, terminal 501 is supplied with a motion vector signal currently inputted.
The signal which has been delivered to the terminal 504 is sent to switcher 503. Further, the currently inputted motion vector signal is sent to common terminal of switch 506 through scale converter 509 for implementing a predetermined scale conversion B to the currently inputted motion vector on the basis of scale indication signal SCB from the switcher 503, and is directly sent to adding input terminal of difference circuit 505.
The switch 506 has selected terminals a to d. Selected terminal a is connected to input terminal of register memory PMV1 of register memory group 502 for storing motion vector which will be described later; selected terminal b is connected to input terminal of register memory PMV2 of register group 502; selected terminal c is connected to input terminal of register memory PMV3; and selected terminal d is connected to input terminal of register memory PMV4. Further, output terminal of register memory PMV1 of register memory group 502 for storing motion vector is connected to selected terminal a of switch 507; output terminal of register memory PMV2 is connected to selected terminal b of switch 507; output terminal of register memory PMV3 is connected to selected terminal c of switch 507; and output terminal of register memory PMV4 is connected to selected terminal d of switch 507. These switches 506 and 507 are of a structure such that any one of the selected terminals a to d is selected in accordance with register switching signal outputted from the switcher 503.
An output of the switch 507 is sent to subtracting input terminal of the difference circuit 505 through scale converter 510 for implementing a predetermined scale conversion A on the basis of scale indication signal SCA from the switcher 503. The difference circuit 505 takes a difference between information from the scale converter 510 and motion vector signal from the terminal 501. The difference obtained by this difference circuit 505 is outputted from terminal 508 as difference motion vector signal from the difference determination element.
Further, information of the currently inputted motion vector is overwritten, through the scale converter 509 and through the switch 506, into any one of register memories PMV1.about.PMV4 designated by register switching signal, and is newly stored thereinto.
As stated above, in the difference determination element for determining difference between motion vectors shown in FIG. 16, register memory group 502 comprised of four register memories PMV1.about.PMV4 for storing the motion vector signal is prepared.
The operation of register memory group 502 for storing motion vector will now be described.
In ISO-IEC/JTC1/SC29/WG11 MPEG93/225b Test Model 4, in the case of P picture, it has been proposed to use respective register memories PMV1.about.PMV4 of register memory group 502 of FIG. 16 in a manner described below.
Namely, for example, information of forward motion vector in which reference picture (picture which serves as reference of predictive picture) is the first field and predicted field with respect thereto (field which is to be coded from now on) is the first field is stored into register memory PMV1. Further, information of forward motion vector in which reference picture is the second field and predicted field is the first field is stored into register memory PMV2. Information of forward motion vector in which reference picture is the first field and predicted field is the second field is stored into register memory PMV3. In addition, information of forward motion vector in which reference picture is the second field and predicted field is the second field is stored into register memory PMV4. Namely, in the P picture, the relationship between motion vector signals and respective register memories in which they are stored was determined by combinations of reference pictures and fields to be predicted.
On the other hand, in ISO-IEC/JTC1/SC29/WG11 MPEG93/225b Test Model 4, in the case of B picture, it has been proposed to use respective register memories PMV1.about.PMV4 of FIG. 16 in a manner described below.
Namely, for example, two register memories PMV are prepared for forward motion vector. For example, information of forward motion vector in which reference picture is the first field is stored into register memory PMV1, and information of forward motion vector in which reference picture is the second field is stored into register memory PMV2. Further, two register memories PMV are prepared for backward motion vector as well. Information of backward motion vector in which reference picture is the first field is stored into register memory PMV3, and information of backward motion vector in which reference picture is the second field is stored into register memory PMV4. Namely, in the B picture, irrespective of whether a field to be predicted is the first field or the second field, motion vector signals in which reference picture and direction of prediction are the same were stored into the same register memory.
In the difference determination element shown in FIG. 16, in allowing the currently inputted motion vector to be difference vector, an approach is employed to take out the value of register memory PMV corresponding to that motion vector (i.e., PMV in which a motion vector having parity of reference picture and predicted field which are the same as the motion vector is stored in the case where that motion vector is used for coding of P picture; and, PMV in which a motion vector having parity of reference picture and direction of prediction which are same as the motion vector is stored in the case where that motion vector is used for coding of B picture) to implement a predetermined scale conversion A by scale converter 510 thereto as occasion demands (in accordance with scale indication signal SCA) thereafter to carry out calculation between the scale converted value A and the above-mentioned currently inputted motion vector by difference circuit 505. Further, scale conversion B is implemented to the value of the currently inputted motion vector by scale converter 509 as occasion demands thereafter to overwrite it into any one of register memories PMV selected and subjected to reference at the time of determination of difference motion vector as described above although not illustrated to newly store it thereinto. Difference determination element is constructed in this way.
FIG. 17 shows an inverse difference determination element corresponding to difference determination element for motion vector of FIG. 16.
In the inverse difference determination element shown in FIG. 17, terminal 524 is supplied with motion compensation mode signal, prediction type signal, picture structure signal, field motion vector parity signal, predicted field parity signal, and picture coding type signal. In addition, terminal 518 is supplied with the currently inputted difference motion vector signal.
Information through the terminal 524 is sent to switcher 523, and the difference motion vector is sent to one adding input terminal of adding circuit 519.
Further, selected terminal a of switch 521 is connected to output terminal of register memory PMV1 of motion vector storage register memory group 520 for carrying out storage similar to that of motion vector storage register memory group 502 of FIG. 16; selected terminal b is connected to output terminal of register memory PMV2; selected terminal c is connected to output terminal of register memory PMV3; and selected terminal d is connected to output terminal of register memory PMV4. This switch 521 is of a structure such that any one of selected terminals a to d is selected in accordance with register switching signal from the switcher 523. Common terminal of this switch 521 is connected to the other adding input terminal of the adding circuit 519 through scale converter 529 for implementing a predetermined scale conversion C to a signal delivered on the basis of scale indication signal SCC from the switcher 123.
Accordingly, the adding circuit 519 takes an addition of motion vector signal passed through the switch 521 from register memory group 520 in accordance with register switching signal outputted from the switcher 523 and further through scale converter 529 and difference motion vector signal from the terminal 518. An addition output of the adding circuit 519 is outputted from terminal 525 as reconstructed motion vector from the inverse difference determination element.
Further, the reconstructed motion vector signal from the adding circuit 519 is sent also to common terminal of switch 522 through scale converter 530 for implementing a predetermined scale conversion D to a signal delivered on the basis of scale indication signal SCD from the switcher 523. This switch 522 is of a structure such that any one of selected terminals a.about.d is selected on the basis of the register switching signal. In this switch 522, selected terminal a is connected to input terminal of register memory PMV1 of the register memory group 520, selected terminal b is connected to input terminal of register memory PMV2, selected terminal c is connected to input terminal of register memory PMV3, and selected terminal d is connected to input terminal of register memory PMV4.
Thus, information of the reconstructed motion vector is overwritten and is newly stored into any one of register memories PMV1.about.PMV4 designated by the register switching signal through the switch 522.
Namely, in the inverse difference determination element of FIG. 17, the value stored in register memory PMV corresponding to inputted difference motion vector (i.e., PMV in which reconstructed motion vector having parity values of reference picture and predicted field which are the same as the difference motion vector is stored in the case where difference motion vector is used for decoding of P picture, or PMV in which reconstructed motion vector having reference picture parity and direction of prediction which are the same as the difference motion vector is stored in the case where difference motion vector is used for decoding of B picture) corresponding to inputted differential motion vector is taken out to implement scale conversion C to that value by using scale converter 529 as occasion demands thereafter to carry out an addition calculation of that scale converted value and the currently inputted difference motion vector by using adding circuit 519, thus to reconstruct motion vectors corresponding to original motion vector. The reconstructed motion vector is caused to undergo scale conversion D by scale converter 530 as occasion demands thereafter to overwrite and newly store the scale converted value into any one of register memories PMV1.about.PMV4 selected and subjected to reference at the time of reconstruction of motion vector. Inverse difference determination element is constructed in this way.
An example of operation in the difference determination element of FIG. 16 at B picture will now be described with reference to FIG. 18.
In FIG. 18, with respect to macroblock (small block) MB1, it is assumed that there is no predictive error signal to be transmitted (the above-mentioned condition b.sub.-- 1e is satisfied). Further, macroblocks MB0 and MB1 are assumed to have the same forward prediction mode (the above-mentioned condition b.sub.-- 2e is satisfied).
At this time, it is assumed that, as the first example, picture structure is field structure and predicted field parity is the first field (predicted field is the first field). The prediction type is 16.times.8 Field based prediction in the case of respectively independently carrying out prediction with respect to the area of 16 pixels.times.8 lines of the upper portion and the area of 16 pixels.times.8 lines of the lower portion. In the example of FIG. 18, for example, motion vector of the upper portion of macroblock MB0 is (5, 1).sub.(1), motion vector of the lower portion of macroblock MB0 is (5, 5).sub.(2), motion vector of the upper portion of macroblock MB1 is (5, 5).sub.(3), motion vector of the lower portion of macroblock MB1 is (5, 5).sub.(4), motion vector of the upper portion of macroblock MB2 is (5, 5).sub.(5), and motion vector of the lower portion of macroblock MB2 is (5, 5).sub.(6).
Further, it is assumed that all motion vectors of macroblocks MB0 and MB1 make reference to the same first field. In this case, since parity of reference picture and direction of prediction are the same with respect to all motion vectors, differences between motion vectors stored in register memory PMV1 of FIG. 16 are taken to transmit difference motion vector. It is assumed that (0, 0) is stored as an initial value in register memory PMV1 of FIG. 16.
Namely, in the case of the example of FIG. 18, in macroblock MB0, (5, 1).sub.(1) -PMV1=(5, 1) which is a difference between motion vector (5, 1).sub.(1) of the upper portion and initial value (0, 0) of register memory PMV1 is obtained. The value thus obtained is outputted as difference motion vector of the upper portion of MB0. The motion vector (5, 1).sub.(1) is stored into register memory PMV1. Further, (5, 5).sub.(2) -PMV1=(0, 4) which is a difference between motion vector (5, 5).sub.(2) of the lower portion and value (5, 1) of register memory PMV1 is obtained. The value thus obtained is outputted as difference motion vector of the lower portion. The motion vector (5, 5).sub.(2) is stored into register memory PMV1. Then, by similarly taking a difference also with respect to macroblock MB1, difference motion vectors all become equal to zero (0, 0) in the macroblock MB1. (The above-mentioned condition b.sub.-- 3e is satisfied).
Further, as the second example, when condition b.sub.d-- 1e and condition b.sub.-- 2e are satisfied as previously described, the picture structure is assumed to be frame structure. The prediction type is assumed to be Field based prediction for respectively independently predicting the first field and the second field of macroblock MB (predicted field is assumed to be the first field and the second field). Further, it is assumed that the first field of macroblock MB0 is (5, 1).sub.(1), motion vector of the second field is (5, 5).sub.(2), motion vector of the second field of macroblock MB1 is (5, 5).sub.(3), motion vector of the second field is (5, 5).sub.(4), motion vector of the second field of macroblock MB2 is (5, 5).sub.(5), and motion vector of the second field is (5, 5).sub.(6).
In addition, all motion vectors of macroblock MB0 and macroblock MB1 are assumed to make reference to the same first field. Also in this case, since parity of reference picture and predictive direction are the same with respect to all motion vectors, differences between motion vectors stored in register memory PMV1 of FIG. 16 are taken to transmit difference motion vector. Thus, also in this example, the method of taking difference is the same as that of FIG. 18. In macroblock MB1, difference motion vectors all become equal to zero (0, 0) (condition b.sub.-- 3e is satisfied).
From facts as described above, in the above-described two examples, in accordance with the rule at the MPEG1, the above-mentioned conditions b.sub.-- 1e, b.sub.-- 2e, b.sub.-- 3e are satisfied in macroblock MB1. Accordingly, it is now assumed that encoder has processed this macroblock MB1 as skipped macroblock.
In this case, however, that is an erroneous selection. This is because, on the decoder side, processing of the above-mentioned conditions b.sub.-- 1d and b.sub.-- 2d are correctly carried out, but copy is carried out although motion vectors of macroblocks MB0 and MB1 are actually different. Namely, if macroblock MB1 is judged to be skipped macroblock, in the case of the former example, although motion vector (5, 1).sub.(1) of the upper portion of macroblock MB0 and motion vector (5, 5).sub.(3) of the upper portion of macroblock MB1 are different (5, 1).sub.(1) .noteq.(5, 5).sub.(3), copy of motion vector from macroblock MB0 to macroblock MB1 is carried out. As a result, decoding is not correctly carried out. Further, also in the case of the latter example, although motion vector (5, 1).sub.(1) of the first field of macroblock MB0 and motion vector (5, 5).sub.(3) of the first field of macroblock MB1 are actually different, an erroneous copy is carried out.
This problem results from the fact that, in the prior art, in B picture, in allowing motion vectors within the same macroblock to be difference motion vector to transmit it, the same register memory (PMV) is used twice or more to update the content thereof.
Further, the fact that the same register memory (PMV) is used twice or more with respect to motion vector within the same macroblock further gives rise to problems as described below on the decoder side.
FIG. 19 shows the operation for reconstructing motion vector from difference motion vector transmitted with respect to B picture. It is to be noted in this example that picture structure is field structure, and macroblocks MB0, MB1, MB2 are encoded in the forward predictive mode respectively independently in the area of 16 pixels.times.8 lines of the upper portion and the area of 16 pixels.times.8 lines of the lower portion. In addition, it is assumed that the area of 16 pixels.times.8 lines of the upper portion and the area of 16 pixels.times.8 lines of the lower portion are caused to be both single first field.
In the example of FIG. 19, since the area of the upper portion and the area of the lower portion have the same parity of reference picture and the same direction of prediction, e.g., only register memory PMV1 is used, so the content of that register memory PMV1 varies within the same macroblock. For this reason, in order to hold motion vector of macroblock processed immediately before, it is necessary to have additional four register memories (respective two register memories for forward and backward vectors) except for the register memories PMV. It is to be noted that the method of holding motion vector of macroblock immediately before is frequently carried out also with a view to carrying out error correction, etc. on the decoder side.
Namely, in FIG. 19, it is assumed that, as difference motion vector obtained from the configuration of FIG. 18, (5, 1).sub.(1) of difference motion vector of the upper portion of macroblock MB0, (0, 4).sub.(2) of difference motion vector of the lower portion of macroblock MB0, (0, 0).sub.(3) of difference motion vector of the upper portion of macroblock MB1, (0, 0).sub.(4) of difference motion vector of the lower portion of macroblock MB1, (0, 0).sub.(5) of difference motion vector of the upper portion of macroblock MB2, and (0, 0).sub.(5) of difference motion vector of the lower portion of macroblock MB2 are obtained. Further, it is assumed that (0, 0) is stored as an initial value in the register memory PMV1.
In the case of the example of FIG. 19, in macroblock MB0, (5, 1).sub.(1) +PMV1=(5, 1).sub.A by addition of difference motion vector (5, 1).sub.(1) of the upper portion and initial value (0, 0) of register memory PMV1 is obtained as reconstructed motion vector. This reconstructed motion vector is sent to register memory PMV1. Further, at the lower portion of macroblock MB0, (0, 4).sub.(2) +PMV1=(5, 5).sub.B by addition of the difference motion vector (0, 4).sub.(2) of the lower portion and reconstructed motion vector into the register memory PMV1 is obtained as reconstructed motion vector. This reconstructed motion vector is sent to register memory PMV1. Similarly, in macroblock MB1, (0, 0).sub.(3) +PMV1=(5, 5).sub.C by addition of difference motion vector (0, 0).sub.(3) of the upper portion and reconstructed motion vector (5, 5).sub.B into the register memory PMV1 is obtained as reconstructed motion vector. This reconstructed motion vector is sent to register memory PMV1. Further, at the lower portion of this macroblock MB1, (0, 0).sub.(4) +PMV1=(5, 5).sub.D by addition of difference motion vector (0, 0).sub.(4) of the lower portion and reconstructed motion vector (5, 5).sub.C into the register memory PMV1 is obtained as reconstructed motion vector. This reconstructed motion vector is sent to register memory PMV1. As stated above, since reconstructed motion vectors thus obtained are sequentially overwritten into register memory PMV1, reconstructed motion vector of macroblock immediately before disappears.
As stated above, on the decoder side, in order to hold the value of motion vector reconstructed in macroblock MB immediately before, it is necessary to additionally have two register memories PMV with respect to one direction of prediction, and to store, e.g., values of the motion vectors (5, 1).sub.A and (5, 5).sub.B in the macroblock MB0 and values of the motion vectors (5, 5).sub.C and (5, 5).sub.D in the macroblock MB1.
As described above, in the conventional method of coding motion vector proposed to the MPEG2, no consideration is made in connection with handling of skipped macroblock. For this reason, this coding method has the problem that it is unable to handle skipped macroblock.
Further, on the decoder side, in order to hold motion vector of macroblock processed immediately before, it is necessary to have register memories in addition to the register memories PMV, thus allowing the configuration of inverse difference determination element for vector (difference motion vector) to be complicated.
In addition, in the prior art, control for switching between register memories PMV at vector difference determination element and/or inverse difference determination element is complicated, resulting in the problem that hardware scale of switcher of register memories PMV is large.